131 research outputs found
Partial convolution based multimodal autoencoder for art investigation
Autoencoders have been widely used in applications with limited annotations to extract features in an unsupervised manner, pre-processing the data to be used in machine learning models. This is especially helpful in image processing for art investigation where annotated data is scarce and difficult to collect. We introduce a structural similarity index based loss function to train the autoencoder for image data. By extending the recently developed partial convolution to partial deconvolution, we construct a fully partial convolutional autoencoder (FP-CAE) and adapt it to multimodal data, typically utilized in art investigation. Experimental results on images of the Ghent Altarpieceshow that our method significantly suppresses edge artifacts and improves the overall reconstruction performance. The proposed FP-CAE can be used for data preprocessing in craquelure detection and other art investigation tasks in future studies
Visibility Aware Human-Object Interaction Tracking from Single RGB Camera
Capturing the interactions between humans and their environment in 3D is
important for many applications in robotics, graphics, and vision. Recent works
to reconstruct the 3D human and object from a single RGB image do not have
consistent relative translation across frames because they assume a fixed
depth. Moreover, their performance drops significantly when the object is
occluded. In this work, we propose a novel method to track the 3D human,
object, contacts between them, and their relative translation across frames
from a single RGB camera, while being robust to heavy occlusions. Our method is
built on two key insights. First, we condition our neural field reconstructions
for human and object on per-frame SMPL model estimates obtained by pre-fitting
SMPL to a video sequence. This improves neural reconstruction accuracy and
produces coherent relative translation across frames. Second, human and object
motion from visible frames provides valuable information to infer the occluded
object. We propose a novel transformer-based neural network that explicitly
uses object visibility and human motion to leverage neighbouring frames to make
predictions for the occluded frames. Building on these insights, our method is
able to track both human and object robustly even under occlusions. Experiments
on two datasets show that our method significantly improves over the
state-of-the-art methods. Our code and pretrained models are available at:
https://virtualhumans.mpi-inf.mpg.de/VisTrackerComment: accepted to CVPR 202
CHORE: Contact, Human and Object REconstruction from a single RGB image
While most works in computer vision and learning have focused on perceiving
3D humans from single images in isolation, in this work we focus on capturing
3D humans interacting with objects. The problem is extremely challenging due to
heavy occlusions between human and object, diverse interaction types and depth
ambiguity. In this paper, we introduce CHORE, a novel method that learns to
jointly reconstruct human and object from a single image. CHORE takes
inspiration from recent advances in implicit surface learning and classical
model-based fitting. We compute a neural reconstruction of human and object
represented implicitly with two unsigned distance fields, and additionally
predict a correspondence field to a parametric body as well as an object pose
field. This allows us to robustly fit a parametric body model and a 3D object
template, while reasoning about interactions. Furthermore, prior pixel-aligned
implicit learning methods use synthetic data and make assumptions that are not
met in real data. We propose a simple yet effective depth-aware scaling that
allows more efficient shape learning on real data. Our experiments show that
our joint reconstruction learned with the proposed strategy significantly
outperforms the SOTA. Our code and models will be released to foster future
research in this direction.Comment: 19 pages, 7 figure
Assisting classical paintings restoration : efficient paint loss detection and descriptor-based inpainting using shared pretraining
In the restoration process of classical paintings, one of the tasks is to map paint loss for documentation and analysing purposes. Because this is such a sizable and tedious job automatic techniques are highly on demand. The currently available tools allow only rough mapping of the paint loss areas while still requiring considerable manual work. We develop here a learning method for paint loss detection that makes use of multimodal image acquisitions and we apply it within the current restoration of the Ghent Altarpiece.
Our neural network architecture is inspired by a multiscale convolutional neural network known as U-Net. In our proposed model, the downsampling of the pooling layers is omitted to enforce translation invariance and the convolutional layers are replaced with dilated convolutions. The dilated convolutions lead to denser computations and improved classification accuracy. Moreover, the proposed method is designed such to make use of multimodal data, which are nowadays routinely acquired during the restoration of master paintings, and which allow more accurate detection of features of interest, including paint losses.
Our focus is on developing a robust approach with minimal user-interference. Adequate transfer learning is here crucial in order to extend the applicability of pre-trained models to the paintings that were not included in the training set, with only modest additional re-training. We introduce a pre-training strategy based on a multimodal, convolutional autoencoder and we fine-tune the model when applying it to other paintings. We evaluate the results by comparing the detected paint loss maps to manual expert annotations and also by running virtual inpainting based on the detected paint losses and comparing the virtually inpainted results with the actual physical restorations. The results indicate clearly the efficacy of the proposed method and its potential to assist in the art conservation and restoration processes
Coupling Efficiency Measurements for Long-pulsed Solid Sodium Laser Based on Measured Sodium Profile Data
In 2013, a serial sky test has been held on 1.8 meter telescope in Yunnan observation site after 2011-2012 Laser guide star photon return test. In this test, the long-pulsed sodium laser and the launch telescope have been upgraded, a smaller and brighter beacon has been observed. During the test, a sodium column density lidar and atmospheric coherence length measurement equipment were working at the same time. The coupling efficiency test result with the sky test layout, data processing, sodium beacon spot size analysis, sodium profile data will be presented in this paper
An optically pumped atomic clock based on a continuous slow cesium beam
Herein, we report the scheme of an optically pumped atomic clock based on a cold cesium atomic beam source. We propose the laser system and physical mechanism of this atomic clock, wherein the atomic beam travels in an upper parabolic trajectory, thereby eliminating the light shift effect. In the experiments, when the length of the free evolution region was 167 mm, the line width of the Ramsey fringe was 37 Hz. When the expected signal-to-noise ratio of the Ramsey fringe that can be achieved is 36,000, the expected short-term frequency stability is about 3.6 × 10–14/√τ, which is significantly higher than that of a conventional optically pumped cesium clock of similar volume
Potential of Core-Collapse Supernova Neutrino Detection at JUNO
JUNO is an underground neutrino observatory under construction in Jiangmen, China. It uses 20kton liquid scintillator as target, which enables it to detect supernova burst neutrinos of a large statistics for the next galactic core-collapse supernova (CCSN) and also pre-supernova neutrinos from the nearby CCSN progenitors. All flavors of supernova burst neutrinos can be detected by JUNO via several interaction channels, including inverse beta decay, elastic scattering on electron and proton, interactions on C12 nuclei, etc. This retains the possibility for JUNO to reconstruct the energy spectra of supernova burst neutrinos of all flavors. The real time monitoring systems based on FPGA and DAQ are under development in JUNO, which allow prompt alert and trigger-less data acquisition of CCSN events. The alert performances of both monitoring systems have been thoroughly studied using simulations. Moreover, once a CCSN is tagged, the system can give fast characterizations, such as directionality and light curve
Detection of the Diffuse Supernova Neutrino Background with JUNO
As an underground multi-purpose neutrino detector with 20 kton liquid scintillator, Jiangmen Underground Neutrino Observatory (JUNO) is competitive with and complementary to the water-Cherenkov detectors on the search for the diffuse supernova neutrino background (DSNB). Typical supernova models predict 2-4 events per year within the optimal observation window in the JUNO detector. The dominant background is from the neutral-current (NC) interaction of atmospheric neutrinos with 12C nuclei, which surpasses the DSNB by more than one order of magnitude. We evaluated the systematic uncertainty of NC background from the spread of a variety of data-driven models and further developed a method to determine NC background within 15\% with {\it{in}} {\it{situ}} measurements after ten years of running. Besides, the NC-like backgrounds can be effectively suppressed by the intrinsic pulse-shape discrimination (PSD) capabilities of liquid scintillators. In this talk, I will present in detail the improvements on NC background uncertainty evaluation, PSD discriminator development, and finally, the potential of DSNB sensitivity in JUNO
Real-time Monitoring for the Next Core-Collapse Supernova in JUNO
Core-collapse supernova (CCSN) is one of the most energetic astrophysical
events in the Universe. The early and prompt detection of neutrinos before
(pre-SN) and during the SN burst is a unique opportunity to realize the
multi-messenger observation of the CCSN events. In this work, we describe the
monitoring concept and present the sensitivity of the system to the pre-SN and
SN neutrinos at the Jiangmen Underground Neutrino Observatory (JUNO), which is
a 20 kton liquid scintillator detector under construction in South China. The
real-time monitoring system is designed with both the prompt monitors on the
electronic board and online monitors at the data acquisition stage, in order to
ensure both the alert speed and alert coverage of progenitor stars. By assuming
a false alert rate of 1 per year, this monitoring system can be sensitive to
the pre-SN neutrinos up to the distance of about 1.6 (0.9) kpc and SN neutrinos
up to about 370 (360) kpc for a progenitor mass of 30 for the case
of normal (inverted) mass ordering. The pointing ability of the CCSN is
evaluated by using the accumulated event anisotropy of the inverse beta decay
interactions from pre-SN or SN neutrinos, which, along with the early alert,
can play important roles for the followup multi-messenger observations of the
next Galactic or nearby extragalactic CCSN.Comment: 24 pages, 9 figure
- …